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[Table 1] 

Description of Reference plane: 
Reference plane: := Model of the ground 



RECEIVED 
MAR (J 2 2004 
Technology Center 2600 



<Ref BD> text 
<Plane> cordinates 
<Metric> array of numeric 



~ Name of reference plane 

Center of the Area (e.g. (0*0)) 

— define transformation (e.g. 3x3 matrix for 
Affine transformation) 



Description of Zone Description: 



Zone Description:™ Define meaningful space on the ground 



<ZoneIB> text 



<Space> space desc 



— Identified the zone on the ground 
(e.g. Goal, Center line) 

— define the space on the ground 



Description of Camera Spec (option!* 



Camera Spec: := Define camera model 



<CameraType> text Identify camera model 

<Param Asrsty> array of numeric define model transformation 
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[Table 2] RECEIVED 
Description of Action: MAR 0 2 2004 

Technology Center 2600 

Acrion;: Describe single player's action 

<Action ID> text Action Symbol (e.g. Run, Kick Walk, etc) 

(text representing types of actions) 

<T-Interval> time interval — ~ Time Interval of this action (represented 

by starting and ending times) 



<Object E» numeric Object Identifier (object of this action) 



<Trajectoiy> Time Stamped Polyline 
(a line with a time attribute of each node) 



- Trajectory of the player in this action 
(trajectory of an object on reference plane 
in the time interval of this action) 



S.N. 09/890,168 

"METHOD AND DEVICE FOR 
DESCRIBING VIDEO CONTENTS" 
Docket No. 954-0 10444-US (PAR) 

Page 8 of 15 RECEIVED 

[Table 3] MAR 0 2 2004 

Description of I Action: Technology Center 2600 

lAction: := Meaningful event in the domain, composed of multiple players and ball 



<!ActionID> text 

<T-Interval> time interval 

<No of Object> numeric 

<Object ID> array of numeric 

<Space> Trajectory 

Polylines, Polygons 



Event Symbol 

(e.g. Shot, Pass, Through Pass, etc) 
(text representing types of events) 

Time interval of this action 

Number of Objects 

Array of Objects identifier 

Spatial description of this action 
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RECEIVED 

MAR 0 2 2004 
Technology Center 2600 
Definition of lAct 

A list of an I Act, a child Act and a child ball (a child I Act and a child Act, etc. mean 
an lAct and an Act comprising this IAct) 

where 

[All the judgment parts and judgment statements with auxiliary functions are linked 
by AND] 

fill 

[An element assignment part of a defined IAct] 
end 



Here, variable symbols in a define statement have respective types which are 
represented by a first character of each variable. The types are as follows. 




Definition of IAct: 
begin 



Type Description 

f a frame 

t a time interval (consisting of starting and ending frames) 

o an object 

0 a group of objects (consisting of a set of objects and the number of 
them) 

p a point (consisting of spatial component x and y) 

P a group of points (consisting of a set of points and the number of them) 

A a group of points, which means a polyline linking them 

1 a time point (consisting of spatial component x, y and time component t) 

L a group of time points (consisting of a set of time points and the number 
of them) 

i an integer 

d a real number 
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Lim 


s Define statement 


Description 


I 


begin 


Beginning. 


2 


tact Through_pass tOOO L0 


Here, it defines an Uct "Through Pass*. 


3 


childJadlPasstlOl LI 


(act pass as one comprising this lacL 


4 


child act 3 Slay Walk Run t2o2 false L2 And an Act of defense side player 1 (H means that a player with object ID o2 






(Stayed, Walked or Ran) moving on trajectory L2 during time interval t2). 


5 
6 


cfaild_ad 3 Stay Walk Run t3 o3 false 13 An Act of defense side player 2. 


7 


where 




8 


greater Jhan o2 o3 


Two players of the defense side have different object IDs (their IDs are o2, o3). 


9 


get_object_fiom_00 o4 1 Ol 


Take the first player of lad ("Pass*) to put in variable o4. 


10 


not_same_team o4 o2 


o4 and o2 are different teams. 


11 


nol_sanie_team o4 o3 


o4 and o3 are different teams. 


12 


set_lengm_of_polyline dO LI 


Length of a pass is measured 


13 


|ess_than dO 20.0 


Length of 20 m or less. 


14 


tem|joral_overiap t2 13 


Two players of the defense side have overlapping time tntervala. 


15 


set^tempoi^^vcrUppingjxriod t4t2t3 


The overlapping time intervals to variable t4. 


16 


teaiporal_overiap tl 14 


There is an overlap in time intervals of t4 and an Iact pass. 


17 


set temporal ovcriapotng period t5 tl t4 Overlapping time intervals (consequently, overlapping Acts and lad passes of me 




two players of the defense side) to variable t5. 


18 


get_fi«me_start_ofj>eriod f0t5 


Put a starling frame of time interval t5 in time point fO. 


19 


get_f>aine_cnd_of_pcriod fl t5 


Put an ending frame of time interval t5 mtnnepaintfl. 


20 


8 et - ST_GL_drvidingj>oim pO L2 fD 


The position (location) of player 1 of the defense side at tone point fO to variable 

po. 


21 


gclS_ST_OL_divkling_point pi L3 10 


The position (location) of player 2 of the defense side at time point fO to variable 
pi. 


22 


8 et^ST_OL_dividing_jK>int p2L2fl 


The position (location) of player 1 of the defense side at time point fl to variable pX 


23 


set ST_OL_divkUngjKMm p3L3fl 


The position (location) of player 1 of the defense skte at tin* point fl to variable p3. 


24 


gei_pomt_from_poryline p4 1 LI 


The starting point of a pass to p4. 


25 


getj>oirt_lroni_polyltne p5 -1 LI 


The ending point of a pass to p5. 


26 


Creale_tine PI 2 p4 p5 


Create a line linking p4 and p5 (a pass course). 


27 


Create_finc P2 2p0pl 


A line linking pO and pi (a line linking the two players of the defense side at time 




point ID). 


28 


Createjine P3 2p2P3 


A line linking p2 and p3 (a line linking the two players of the defense side at time 




point fl). 


29 


spatsaljcross PI P2 


PI and P2 arc crossing. 


30 


8C l_d^stance_pxrint_and_line dl pO p4 p5 


The distance from pass course pO at time point fO is soughL 


31 


set distance_j>otnl_and_line d2 pi p4 p5 


The distance from pass course pi at time point ID is sought 


32 


lessjhan dl 7.0 


The distance is 7 m or less. 


33 


lessjhan d2 7.0 


Same, 


34 


spatial.cross PI P3 


PI end P are crossing. 


35 


set distance _point_and_line d3 p2 p4 p5 


The distance from a pass course at time point fl is sought as above. 


36 


set_distanoe jx>mt_andjine d4 p3 p4 p5 




37 


less Jhan d3 7.0 


The distance is 7 m or less. 


38 


lessjhan d4 7.0 




39 


fill 


If the above conditions are met, 


40 


tOtl 


Assign the time interval of tl (lad pass) to tO (the time interval of an Iact through 


P*»). 


41 


O0O1 


Assign the group of objects of Ol (Iact pass) to Ol (the group of objects of an Iact 




through pass). 


42 


L0L1 


Assign LI (a trajectory of lad pass) to LO (spatial representation of lad through 


pass). 


43 


end 


End. 



[Table 5] 
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[Table 6] 



Line Define statement 



1 pass: 

2 begin 

3 Iact Pass tO 00 LO 

4 Childact Kick] Jump| Sliding tl bl LI 

5 Child_act Run|Stay|Walk t2 o2 L2 

6 Ball t3 L3 

7 where 

8 same_team(ol,o2) 

9 get_frame_start_peri od fl t3 

10 get_frame_end_peri od £2 13 

1 1 temporal overlap tl t3 

12 temporaloverlap t2t3 

13 set_S T_GL_dividing_point pi fl L3 

14 set_S T_GL_dividing_point p2 £2 L3 

15 set_S T_GL_dividing_point p3 fl LI 

16 set_S T_GL_dividing_point p4 £2 L2 

17 set_distance_point_to_point dl pi p3 

18 set_distance_point_to_point d2 p2 p4 
Lessthan dl 0.5 
Lessthan d2 0.5 
set_Go_from_obj ects Ol 2 ol o2 
fill 
t0t3 
L0L3 
O0 Ol 
End 



RECEIVED 

MAR 0 2 2004 
Technology Center 2600 



19 
20 

21 
22 
23 
24 
25 
26 
27 
28 
29 



long pass: 
begin 

30 iact Long_pass tO OO LO 

31 chikUact Pass tl Ol LI 

32 where 

33 set_length_of_j>olylbe dO LI 

34 Greater Than dl 30.0 

35 fiU 

36 tOtl 

37 O0O1 

38 L0L1 

39 end 
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[Table 7] 

Line Define statement 



1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 



feed pass: 
begin 

Iact Feed Pass tO O0 LO 
Child_Iact Pass tl Ol LI 
Child_act Run t2 o2 L2 
where 

get_object_from GO o3 -1 Ol 

same_object o2o3 

temporalduring tl t2 

fill 

tO t l 

O0O1 

L0L1 

end 



RECEIVED 

MAR 0 2 2004 
Technology Center 2600 



cross pass: 
begin 

Iact Cross pass tO 00 L0 
Childjact Pass tl Ol LI 
where 

get_frame_start__period fl tl 

22 get_frame_end_period f2tl 

23 setJST_GL_dividing_point pi fl LI 

24 set ST GL dividing joint p2 £2 LI 

25 setjength^ofjjolyline dl LI 

26 set_X_distance_point_and_point d2plp2 

27 greater than dl 30.0 

28 lessjhan d2 5.0 

29 fill 

30 tO tl 

31 O0O1 

32 L0L1 

33 end 



S.N. 09/890,168 
"METHOD AND DEVICE FOR 
DESCRIBING VIDEO CONTENTS" 
Docket No. 954-0 10444-US (PAR) 
Page 13 of 15 

[Table 9] 

Line Define statement 

1 wall pass: 

2 begin 

3 iact l-2_passt0O0L0 RECEIVED 

4 child iact Pass tl Ol LI 

5 childjact Pass t2 02 L2 MAR 0 2 2004 

6 child_act Stay Walk Run t3 ol L3 

7 where Technology Center 2600 

8 get_object_from_GO o2 1 Ol 

9 get_object_from_GO o3 -1 Ol 

10 get_objectJrom_GO o4 1 02 

11 get_pbject_from_GO o5 -1 02 

12 not_same_team ol o2 

13 same_object o2 o5 

14 same_object o3 o4 

15 set_temporal_distance_period il tl t2 

16 Less_Thanil 5 

17 not_same_team ol o2 

18 set_temporal_concatination_period t4 tl t2 

19 temporal_during t3 t4 

20 get_frame_start_of_period fl t4 

21 get_frame_end_ofj>eriod f2 tl 

22 get_frame_end_ofj>eriod f3 t4 

23 set_ST _GL dividing_point pi LI fl 

24 set ST GL dividing jsoint p2 LI £2 

25 setJST_GL_dividing_point p3 L2 £3 

26 Create_Area Al 3 pi p2 p3 

27 set_ST_C3L_dividingj>oint p4 L3 fl 

28 set_ST_GL_dividing_point p5 L3 £3 

29 spatial_j>oint_in p4 Al 

30 spatial _point_in p5 A 1 

31 set_ST_GL_concatinate_polyline L4 LI L2 

32 fill 

33 t0t4 

34 O0 Ol 

35 L0L4 

36 end 
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[Table 10] 

List of group of auxiliary functions: 

(1 ) Functions suitable for soccer 
same team ( PlayerO, Player 1 ) 
samejeam ( Player, Location ) 

It shows whether PlayerO and Player 1 belong to the same team. 
One of them may represent a position such as "GOAL." 

(2) Functions that generally hold apart from soccer 



# 


[Temporal] 




1 


get_frame_start_of_pericd (fl,tl) 


Assign the starting point of time interval tl to fl. 


2 


get_frame_end_of_period (fl.tl) 


Assign the ending point of time interval tl to fl. 


3 


set Dericd from frames f 1 1 fl fl} 


Create time interval tl from two time noints fl fl 


4 


ge*_period_ofJ3L (tl ,L1) 


Assign the lifetime interval of trajectory LI to time interval tl . 


5 


temporal_in (fl.tl) 


Time point fl is inside time interval tl. 


6 


temporal_meet (tl,t2) 


tl and t2 exist in this order, and ending point of tl and starting point 
of t2 are the same. 


7 


temporaJ_overlap (tl,t2) 


There is a time interval overlapping time intervals tl and t2. 


8 


temporal_start (tl ,t2) 


Time intervals tl and t2 have the same starting point 


9 


temporal_finish (tl,t2) 


Time intervals tl and t2 have the same ending point 


10 


temnoral durins ftKt2^ 


Time interval tl is conroletelv included in time infeETV&il t2 

4V 1 1 i IV V AAA I 4 4vw ^rt#**4**#»%**4V J 4V4*W**«*»^^» AAA * fl ■ II nv AAA VMS VHM AO • 


11 


temporal equal (tl,t2) 


Time intervals tl and t2 have the same starting and ending points. 


12 


temporal_before (tl,t2) 


Time interval tl ends earlier than the starting point of t2. No 
overlapping time interval. 


13 


set temporal overlapping_period 
(tl^O) 


Assign the overlapping time intervals of t2 and t3 to tl. 


14 


set temporal conca lination_period 
(tl,t2,t3) 


Assign the concatenated time intervals of t2 and t3 to tl. 


15 


set temporal distance jjcriod 
(ilAt3) 


Assign the difference between ending point of time interval t2 mad 
starting point of t3 to il. 


16 


set ST GL dividingjecus 
(Ll"tUL2) 


Assign to LI the trajectory of a part applicable to partial interval! tl of 
the lifetime interval of trajectory L2. 


17 


set ST GL concatinate locus 
(Ll,L2i3)~ 


Assign to LI the trajectory concatenating trajectories L2 and L3. 




Spatial] 




18 


set ST GL dividing point 

(puun) ~ 


Assign the position of trajectory LI at time point HD to pL 


19 


set_point_from_locus (p 1 ,1 1 ) 


Convert a point 11 of trajectory data to position data pi. 


20 


set_j)oint(pl,il42) 


Define position pi of which x, y coordinates are il , i2. 


21 


Create_Polyline (Pl,il,pl,p2,...) 


Create line PI linking point sets made up of pi, p2, ...(the number, 
il). 



- RECEIVED 
MAR 0 2 2004 
Technology Center 2600 
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[Table 11] 

(2) Functions that generally hold apart from soccer (continued) 



RECEIVED 
MAR 0 2 2004 

Technology Center 2600 



22 



23 



24 



25 



26 



27 



28 



29 



[Temporal] 



Create_Area (Al.il.plja,...) 



Create polyline A1 linking point sets made up of pi, p2, ...(tike 
number, il). 



setJength.cQolyline (dl,Xl) [Assign the length of the line shown by XI to dl. XI as Post L. 



set_deistance_point and_pomt 
(dl.pl.p2) 



Assign the Euclid's distance between positions xl and x2 to dl . x is p 
or 1. 



set_X_distance_point and_j*oint 
(dl.xl.x2) 



Assign the distance on axis x between positions xl and x2 to dl. x 
por 1. 



set_Y_dislance_point and_poimt 
(dl,xl,x2) 



Assign the distance on axis y between positions xl and x2 to dl . x is 
pari. 



serdistance _j*omt_andJine 



Assign the distance between position xl and line PI to dl. xio.por I 



set_distance _jK>int_and_Area 
WMU1) 



Assign the distance between position xl and polyline Al to dl. x is p 
I or 1. 



spatial_point_in (pl,Xl) 



Position pi is included in XI. X is P or A. 



30 spatial Jroe_in (P1.A1) 



I line PI is completely included in polyline Al. 



31 spatiaTapart (P1.P2) 



32 



35 



36 



37 



38 



39 



40 



41 



42 



43 



45 



| No overlap of two lines pi and p2. 



Spatial Jfae_touch (PI .XI) 



I There is a shared point between PI and XI. X is P or A. 



spatsal_through (P1.A1) 



PI is penetrating polyline Al . 



spatial_overlap (A1.A2) 



[ Two polylines Al & A2 are partly overlapping. 



spatial^contain (A1.A2) 



Al is completely included in A2. 



spatial_area_touch (A1.A2) 



[Al and A2 are touching at a point or a line. 



spatial_disjoint (A1.A2) 



No shared part between Al and A2. 



spatial_cross (PI ,P2) 



[Two lines PI and P2 are crossing. 



[Object handling] 



sasne^object (ol ,o2) 



[Two objects ol & o2 are the same objects. 



not_same_object (ol.o2) 



get_c^ectJrom_GO (ol,il,0] ) 



set_GO £rczn_object£j 
(OUl.oI,o2,...) 



get_number.from.GO (il.Ol) 



Numerical) 



greatterjfcan (xl j&) 



less_than(xl,x2) 



46 equal (xl,x2) 



Two objects ol & o2 are different 



i 1-th element of set of object Ol is ol. 



Create object set Ol from ol, o2.... 



To il seeking the number of elements of object set Ol . 



■xl > x2, x is £ d, o. 



xl <x2,xistd,o. 



xl = x2, x is f, d. o. 



